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ABSTRACT 

The paper identifies some major issues in assessing 
creative problem solving. Four main problems are discussed: (1) no 

single widely accepted theory of creativity exists; (2) there ^ is a 
lack of understanding of the implications of the differences in 
assessment procedures; (3) researchers, on the one hand, view 
creativity as entirely a cognitive process or, on the other hand, 
entirely as a complex set of personality traits; and (4) some 
difficulties of assessing creative problem solving relate to the 
heterogeniety of the tasks that have been employed. Also listed are 
number of practical problems, such as: (1) how do we know that what 
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Problems in the Assessrnent of Creative Problem Solving 

DonaW. J. Tref finger 
Purdue University 

Our title for this sym.posium suggests that there has been some 

identifiable progress in the psychologist'' s understanding of creative 

problem solving and its assessment during the Ijist t'wo decades* Looking 

at Eas5.ik''a (1965) bibliography* the Journal of Creative BeMvior^s 

attempts to update it, and at current issues of Psychological Abstracts 

and Dissertation Abstracts * one certainly hopes this is true, given the 

great amount of attention creati^/ity and creative problem eolving continue 

to receive. task, however, is to be the skeptic. It is an easy task, 

for there are a great* many problems which remain unsolved. In this 

brief presentation, I can merely identify some major issues# 

The greatest problemi it seems, is that there is no 3.ingXe# vridely' - 

accepted theory of creativity which directs our efforts. Medni ck ^ s 

work (1962) il3.ustrates , perhaps best of any, the formulation of a theory 

of creativity out of which soms particular method of assessment emergen. 

Yet, for a number of reasons (of., Jackson and Messiok, 1965* Taft and 

Rossiter, 1963) this theory has not been attractive to some researchers, 

and can hardly be considered "wide3-2/'"-aecepted«“ Guilford's structure of 

/ 

intellect model (196?), perhaps best called a theory of intelligence, is 

not a, theory of creativi.ty, despite the fact that it has been heuristically 

and conceptually useful in, describing acme cognitive abilities related to 

creativity. Torrance's tests (1966) purport to be broadly eclectic, drawing 

1. A presentation in the Symj>osium, ^‘Assessing creative. ty: progress 

in both directions." Annual Meet.ing of the American Educational 
Research Association, Minneaj>olis, March 1970. 
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from the ’‘best of the theory available” at the time of their publicationj 
as such, they do not stem from a comprehenBive , xxnified theoretical base. 

From the existiiVi array of ideas about creativity , and from the laCK 
of "theoretical unity," it is no great surprise that there is a profusion 
of tests, all purporting to be measures of "creativity*" Each Instrument 
reflects the pa.rticular set of beliefs and preconceptions of its developer 



about the nature of creativity. 

An out growth of this problem , and a major .isau a in its ovm. rlRht 

is that „ we do not understand very completely... the 

in assessment procedures . Variations in working time, test atmosphere, 
and directions given to the examinee, ‘or example, seem to yield different 



kinds of results, and differing patterns of intercorreXations between 



creativity scores and other cognitive or achievement variables. It is 
quite clear, from the work of W'allach and Kogan (19^5) others, b>ia,t 
such changes occur* It is not clear why, or under what conditions certain 
results might be predicted, Hata we have gathered at Purdue (Van 
Mondfrans, Peldhuseti, Treffinger and Ferris, 1970) suggests that the 



story is much more complex thian merely removing the "ap{)earan,oe of testing" 
and time limits* Removal of time limits, for examp3„6, had no significant 
effect on pupil performance on va: oal tasks. The highest scores on these 
tasks were obtained xmder standard, "test-like" conditions. On figural 
tasks, however, removing time liirdts did influence pupil performance. 
Highest scores were obtained by pf^ipils in a "Take Home" condition* We need 



continuing experimsntal work to und.erstand the problems of test procedures 
and their implications r:x>re completely* Such research would also profit, 
if predictions could be derived from a specific theoretical conception of 
creativity. In the meantime, a clear implication seems to be that 
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researchers who utilize "creativity tests" should be extremely careful 
to report the procedures for te ;t administration, directions, and timing 
in detail. 

Another problem is tlmt many researchers h^ivc tended, on the one hand* 
to view creatiyity as entirely a cognitive proces s* or, on the other hand * 
entirely as a complex set of per3om.lity traits . The former iiave tended 
to ignore the isossibility tliat there iruiy be an affective component to 
creativity, and the latter have tended to overlook the importance of 
underlying cognitive abilities in creative problem solving. A general, 
widely-accepted theory of creativity would, of necessity, consider both 
cognitive and affective factors, and would lead to a more adequate 
assessment procedure. In the meantime, we mast be very cautious about 
our willingness to make inferences about "Creativity" from measures wiiich 
are distinctly cognitive, particularly the divergent-thinking-type tests. 
This does not Imply rejection of the xisefulness of tests of divergent 
thinking. It may be ttiB,t some of the critics have been too severe (e.g. 
Covington, 19^^8; Wallach, 1968). VMle divergent thinking measures 
certainly do not tell the entire story aboxit creativity, these measures 
do very likely assess intellectual abilities which play an Important 
role in creativity* ¥iew:lng creativity as a complex kind of human problem 
solving (which perhaps makes the term creative problem solving preferable), 
divergent thinki.ng is a necessaiy, but not a sufficient, component. 

If this is the case, one might expect certain correilational patterns 
among problem sol->dng tasks which measure creative problem solving. First, 
we might expect that such tasks would be signi.ficantly correlated with 
divergent thinking measures. Secondly, since all problem solving tasks 
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may have in common some correlation with other abilities^ such as 
cognition, memory, and' evaluation, w© would expect a given task to be 
correlated with measures of these abilities. Fina3.1y, we would expect 
that the correlation between the problem solving task and divergent 
thinking would not simply be a function of the relation between divergent 
thinking and the other comraon abilities. That is, we would e-ipect that 
divergent thinking would be signi.ficantly correlated with the problem 
solvi.ng task even when other abilities, correlated with both problem 
solving and divergent thinking, were partialled out. 

We have beer, working recently with multi-solution anagrams problems 
among elementary school children* We have found that these problems 
correlate moderately and positively with fluency, flexibility, originality, 
and IQ, which seems to represent a measure of abilities other than divergent 
thinking. When the effects o.f IQ are partialled out, the divergent 
thinking scores are still sigrdficantly and positively correlated with the 
problem solvi.ng task* In other words, in support of otir expectation, the 
multi-solution anagrams tasks are related to divergent thinking uniquely, 
but are also correlated with measures of other abilities. 

There is also some reason to believe that some of the problems of 
assessing cx*eatlve problem solving relate to the heterogenity of the tasks 
that have been employed . As my colleagijie on this ayn^sium, Gary Davis, 
pointed out (1966), the literature on problem solving is very confusing* 
Tasks have been used in one study and then never again. Some people have 
attempted, as Gary did, to categorize or classify such tasks, but' this has 
tended to be rational rather than empirical. Some new factor analytic data 
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we have just analyzed^ suggests ttet some logical groupings or judgments 
about tasks may not hold up very -w.;!! under closer examination. Some 
tasks, which "on the face" seesa to be attractive measures of creative 
pi’oblem solving reflect quite different ap^pearancc'S empirically. 

For example, our "antelopes" problem, (naking vj-ords from the letters 
in antelopes) would seem, on iihe surface, to be a measure of verbal 
fluency. It loaded si-gnificantly, however, on a factor which aivso 
imi'olved IQ and arititnetic computation ability, but neither of two verbal 
fluency measures* A problem which had no solution, and another prob3.em 
with a limited and fixed nuitiber of solutiorus also I.oaded significantly 
cn the same factor, Mc.lti«»solution anagrams tasks loaded significantly 
on o factor which also included several measures of verbal fluency, 
flexibility, and originality, as well as iQ (to a lesser e,xtent). Two 
problems which had ordy one correct solixtion for each were not related 
significantly to the factor wliich included the aiver,gent thinking measures 
or to the factor which Included “i Q and arithmeti c skills . 

We have a great deal to learn abotrt the assessment of creative problem 
solving. It is qiLite clear that simple n^asures of fluency, flexibility, 
and originality are not sufficient. Perfiaps we must give substant'^al 
effort to finding new, more complex mes,aures*, in this regard, the use of 
ffiulti~sol'u':iion anagrams tasks seems promisi.ng, at least with cidldren. 

Perhaps we mst begin to look more carefully at the interactions of divergent 
thinking scores (fluency-~.fl.exiblH,ty i.nteractions, for example): very little 
use of such combined subscores seems to have been made* 
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Finally^ we have a number of problems of a very practical nature to 
solve. How do we know that what considf^r creative tasks are creative 
and challenging for the examinee? It may be that our most unusual tasktj 
are boring^ tmexciting, even trivial, for the most imaginative of our 
examinees. Perhaps each task that purports to be an assessnxjnt of creati'/e 
problem solving should be accompanied by a simple rating scale: “Have 

you ever worked on this problem before? Did you solve it? V/ere you 
given the solution? What did you think at»out the problems you have solved 
here? Were they interesting? Challenging? What did you think of your 
solutions?" Although I scrupulously avoid using the tern intr*ospection, 
it TTiay be that we could learn quite a bit about our measures by asking 
our subjects to talk about their experiences. 

Perhaps the goodness of a creative response, as Jackson and Me? sick 
have proposed, and the extent to which a problem "captures" the subject, 
as Covington has discussed, are important concerns, but only able to be 
assessed by the subject himself. 

The problems are numerous, an? the answers are fewj even these few 
ideas are tentative and, offered in the spirit of playful inquiry. Solving 
them will be, most likely, half the fun. 
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